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METHOD FOR REPRESENTING A DIGITAL COLOR 

IMAGE USING A SET OF PALETTE COLORS 
CROSS REFERENCE TO RELATED APPLICATIONS 

Reference is made to commonly assigned U.S. Patent Application 

5 Serial No. _ s S4ed entitled "Method For Representing s 

Digital Color Image Using a&€^tof Palette Colors Based on Detected Important 
Colors" by Jiebo Luo et al., the discfcsqre of which is incorporated herein by 
reference. 

FIELD OF THE INVENTION 

10 This invention relates to the field of digital imaging, and more 

particularly to a method for representing a digital color image using a limited 
palette of color values. 

BACKGROUND OF THE INVENTION 
Many color image output devices are not capable of displaying all 

15 of the colors in an input digital image due to the fact that they must be stored in a 
memory buffer with a reduced bit-depth. Likewise, it may also be desirable to 
represent an image using a reduced bit-depth in order to reduce the amount of 
bandwidth needed for the transmission of an image, or the amount of memory 
needed to store an image. For example, many computers may use an 8-bit or a 

20 16-bit color representation to store an image that is to be displayed on a soft-copy 
display such as a CRT or an LCD screen. Such representations allow only 256 
and 65,536 unique color values, respectively. This is significantly less than the 
16,777,216 possible color values associated with a typical 24-bit color image that 
is conventionally used in many digital imaging applications. 

25 In applications where it is necessary to represent an input image 

using a reduced number of colors, it is necessary to determine the set of colors to 
be included in the reduced set of colors. In some cases, a reduced set of colors 
may be determined ahead of time independent of the particular image being 
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encoded. For example, 3 -bits of color information (8 different levels) may be 
used for the red and green channels of an image, and 2-bits of color information 
(4 different levels) may be used for the blue channel of an image. This produces a 
lattice of 8x8x4=256 different color values that can be used to represent the input 
image using an 8 -bit representation. The input digital image can be converted to 
the 8-bit representation simply by taking the highest 2- or 3-bits of each of the 
corresponding RGB channels. The result is an image which has quantization 
errors that can produce visible contours in the image in many circumstances. One 
disadvantage of this method is that any particular image may not contain colors in 
all parts of color space. As a result, there may be some of the 256 color values 
that never get used. Consequently, the quantization errors are larger than they 
would need to be. 

One method for minimizing the visibility of the quantization errors 
in the reduced bit-depth image is to use a multi-level halftoning algorithm to 
preserve the local mean of the color value. (For example see: R. S. Gentile, E. 
Walowit and J. P. Allebach, "Quantization and multilevel halftoning of color 
images for near original image quality," J. Opt. Soc. Am. A 7, 1019-1026 (1990).) 

Another method for minimizing the visibility of the quantization 
errors in the reduced bit-depth image is to select the palette of color values used to 
represent each image based on the distribution of color values in the actual image. 
This avoids the problem of wasting color values that will never be used to 
represent that particular image. Examples of such image dependent palette 
selection methods include vector quantization schemes, such as those described in 
R. S. Gentile, J. P. Allebach and E. Walowit, "Quantization of color images based 
on uniform color spaces," J. Imaging Technol. 16, 11-21 (1990). These methods 
typically involve the selection of an initial color palette, followed by an iterative 
refinement scheme. Another approach, described in R. Balasubramanian and J. P. 
Allebach, "A new approach to palette selection for color images," J. Imaging 
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Technol. 17, 284-290 (1991), starts with all of the colors of an image and groups 
colors into clusters by merging one nearest neighbor pair of clusters at a time until 
the number of clusters equals the desired number of palette colors. A third class 
of vector quantization algorithms uses splitting techniques to divide the color 
space into smaller sub-regions and selects a representative palette color from each 
sub-region. In general, splitting techniques are computationally more efficient 
than either the iterative or merging techniques and can provide a structure to the 
color space that enables efficient pixel mapping at the output. (For a description 
of several such splitting techniques, see commonly assigned U.S. Patent 
5,544,284.) 

While vector quantization mechanisms can yield high quality 
images, they are very computationally intensive. A sequential scalar quantization 
method is set forth by Allebach et al. in U.S. Patent 5,544,284. This method 
sequentially partitions a histogram representing the distribution of the original 
digital color image values into a plurality of sub-regions or color space cells, such 
that each partitioned color cell is associated with a color in the output color 
palette. This method has the advantage that it is generally more computationally 
efficient than vector quantization schemes. 

Image dependent palette selection methods have the significant 
advantage that they assign the palette colors based on the distribution of color 
values in a particular digital image. Therefore, they avoid the problem of having 
palette color values that never get used for a particular image' The various 
methods will generally tend to select palette colors that are representative of the 
most commonly occurring colors in the particular image. This has the result of 
reducing the average quantization errors throughout the image. However, in some 
cases, there may still be large quantization errors in important image regions. For 
example, consider the case where an image contains the face of a person that only 
occupies a small image region. The number of pixels in the image that represent 
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skin-tone colors may be relatively small, and therefore the likelihood that palette 
colors get assigned to skin-tone colors will be low. As a result, when the image is 
represented by the set of chosen palette colors, there may be objectionable 
contours in the face. Since this image region may be very important to an 
observer, these artifacts may be much more objectionable than they would have 
been if they had occurred in other image regions. Other types of image content 
where quantization artifacts may be particularly objectionable would include 
neutral image regions, and blue sky image regions. Existing image dependent 
palette selection techniques do not provide any mechanism for minimizing the 
quantization artifacts in these important image regions unless they are large 
enough in size to comprise a significant portion of the distribution of image 
colors. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide an improved 
method for palette selection which minimizes quantization artifacts for selected 
important colors. 

This object is achieved by a method for converting an input digital 
color image having a set of possible input colors to an output digital color image 
having a set of palette colors, the number of palette colors being less than the 
number of possible input colors, wherein the set of palette colors is determined 
based on the distribution of colors in the input digital image supplemented by a 
distribution of important colors. This is accomplished using the steps of 
determining the distribution of colors in the input digital color image, 
supplementing the distribution of colors by a distribution of important colors, 
determining the set of palette colors to be used in the formation of the output 
digital color image responsive to the supplemented distribution of colors, and 
forming the output digital color image by assigning each color in the input digital 
color image to one of the colors in the set of palette colors. 
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ADVANTAGES 

The present invention has the advantage that the set of palette 
colors that are determined will emphasize important colors whether or not the 
important colors occupy a large area of the input image. The present method 
provides more esthetically pleasing images to a viewer. 

It has the further advantage that any conventional image-dependent 
palettization algorithm, such as sequential scalar quantization or vector 
quantization, can be used in accordance with the present invention by simply 
appending additional pixels to the input image, where the color of the additional 
pixels is distributed according to the distribution of important colors. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a flow diagram illustrating the method of the present 

invention; 

FIG. 2 is a diagram showing the process of appending additional 
pixels to the input digital color image; 

FIG. 3 is a diagram showing a multi-level vector error diffusion 

algorithm; and 

FIG. 4 shows an example set of error weights that can be used for a 
multi-level vector error diffusion algorithm. 

DETAILED DESCRIPTION OF THE INVENTION 

A flow diagram illustrating the basic method of the present 
invention is shown in FIG. 1. The method operates on an input digital color image 
10 having a set of possible input colors. The set of possible input colors will be 
defined by the color encoding of the input digital color image 10. Typically, the 
input digital color image 10 might be a 24-bit RGB color image having 
2 24 =1 6,777,2 16 different colors. However, it will be recognized that the invention 
is not limited to this configuration. Alternatively, the input digital color image 10 
could be at some other bit-depth, or could be in some other color space such as 
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YC R C B or CIELAB. A determine distribution of input colors step 1 1 is used to 
determine the distribution of input colors 12. In a preferred embodiment of the 
present invention, the distribution of the input colors is determined by forming a 
three-dimensional histogram of color values. 

Next, the distribution of input colors 12 is supplemented by a 
distribution of important colors 1 3 to form a supplemented distribution of colors 
14. A determine set of palette colors step 15 is then used to determine a set of 
palette colors 16 responsive to the supplemented distribution of colors 14. Since 
the supplemented distribution of colors 14 has been supplemented by the 
distribution of important colors 13, the set of palette colors will contain more 
colors in and/or near the important color regions than would otherwise be the case 
if the determine set of palette colors step 15 had been applied to the original 
distribution of input colors 12. The number of colors in the set of palette colors 16 
will be less than the number of possible input colors. In a preferred embodiment 
of the present invention, the number of palette colors will be 256 corresponding to 
the number of different colors that can be represented with an 8 -bit color image. 
However, it will be obvious to one skilled in the art that the method can be 
generalized to any number of palette colors. For example, if the output image 
were a 4-bit color image, the number of corresponding output colors would be 16, 
or if the output image were a 10-bit color image, the number of corresponding 
palette colors would be 1024. 

Once the set of palette colors 16 has been determined, an assign 
palette color to each image pixel step 17 is used to form an output digital color 
image 18. The output digital color image 18 will be comprised entirely of colors 
chosen from the set of palette colors. Generally, the palette color for each pixel of 
the image will be identified by an index value indicating which palette color 
should be used for that pixel. For example, if there are 256 palette colors used for 
a particular image, each pixel of the output image can be represented by an 8-bit 
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number in the range 0-255. The output digital color image 18 will generally be 
stored in a digital memory buffer, or in a digital image file. In order to properly 
display the image, a palette index indicating the color value for each of the 
different palette colors needs to be associated with the image. When the image is 
displayed, the palette index can be used to determine the corresponding color 
value for each of the palette colors. 

The steps in the method of FIG. 1 will now be discussed in more 
detail. The determine distribution of input colors step 1 1 can take many forms. In 
a preferred embodiment of the present invention, a three-dimensional histogram of 
the input colors of the input digital color image is computed. One way that the 
histogram of input color values can be computed is to go through each pixel of the 
input image and count the number of occurrences of each input color. To limit the 
number of bins in the histogram of input colors, ranges of input color values can 
be grouped into a single bin. For example, instead of 256 different bins for the red 
dimension, a smaller number of bins such as 32 or 64 could be used. 

Additionally, to speed up the computation of the histogram of input 
colors, it may be desirable to sub-sample the pixels in the input image. For 
example, instead of examining every image pixel, only the pixels in every 10 th row 
and every 10 th column could be used to form the histogram of input colors. In 
some cases, it might be desirable to convert the image to some other color space 
before determining the histogram of input colors. For example, an RGB image 
could be converted to a YC R C B luminance-chrominance representation. This is 
advantageous for some types of palette determination algorithms. This also makes 
it possible to use different bin sizes for the luminance and chrominance color 
channels. 

There are many examples of sets of important colors that could be 
used with the method of the present invention. Generally, the important colors 
should be chosen or selected to be colors that are of high importance to a human 
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observer. Colors that are likely to appear in smoothly varying image regions are 
particularly critical since they will be most likely to suffer from image quality 
degradations resulting from quantization errors. Skin-tone colors are an example 
of colors that might be included in the set of important colors. Not only are skin- 
5 tone colors very important to image quality as judged by a human observer, but 
additionally, they usually occur as slowly varying gradients in an image. There 
are many different variations of skin-tone colors corresponding to different races 
and complexions, as well as to variations in the illumination characteristics of the 
scene. In this case, the distribution of important colors should generally reflect the 
_ 10 expected variations in the skin-tone colors. The distribution of skin-tone colors 

*y can be determined by measuring color values for a wide variety of races, 

O complexions and illuminations, and creating a histogram, or finding some set of 

yl statistical parameters describing the shape of the color distribution. Alternatively, 

ffii 

*gr§ a vector quantization algorithm, such as the well-known "k-means" algorithm 

q 15 could be used to cluster the measured skin-tone color values into a smaller set of 

*2 s representative color values. 

z: Other colors that might be included in the distribution of important 

colors for some applications would be neutral colors and sky colors. In many 
applications, quantization errors in these regions might be particularly visible 

20 and/or objectionable. 

There are several methods that could be used to determine the 
supplemented distribution of colors 14. For example, a histogram of important 
colors could be pre-computed and then combined with the histogram determined 
to represent the distribution of input colors. In this case, the supplemented 

25 distribution of colors 14 could be determined by simply adding the histogram of 
important colors to the histogram of input colors to form a supplemented 
histogram. Alternatively, a weighted combination of the histogram of important 
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colors and the histogram of input colors could be used to adjust the relative weight 
assigned to the two histograms. 

Another approach for determining the supplemented distribution of 
colors 14 is to append additional pixels to the input digital color image to form an 
enlarged input digital color image. The colors of the additional pixels that are 
appended to the image are distributed according to the distribution of important 
colors. The supplemented distribution of colors can then be determined by simply 
determining the distribution of colors in the enlarged input digital color image. 
The additional pixels can be provided in the form of a predetermined target image 
that can be appended to the image. Generally, it may be desirable to resize the 
target image before appending it to the input digital color image. For example, it 
will typically be more convenient to combine the two images if the width of the 
target image is adjusted to match the width of the input digital color image. 

This method is illustrated in FIG. 2. An input image 20 is shown 
which includes a person occupying a relatively small region of the image. A 
conventional palettization method would assign very little weight to the skin-tone 
colors since they make up a statistically small portion of the image. However, the 
colors in the input image are supplemented by appending additional pixels 22 to 
the image to form an enlarged image 24. The colors of these additional pixels 
correspond to the distribution of important colors. For example, if the important 
colors were specified to be skin-tone colors, the additional pixels could include a 
wide variety of skin-tone colors corresponding to different races, complexions and 
illuminations of typical skin-tones. The additional pixels could be discrete 
patches of different skin-tones, or could include smooth gradients covering the 
range of expected skin-tone colors. The supplemented distribution of colors 14 
can then be computed directly from the enlarged image 24, rather than having to 
combine the distribution of input colors 12 and a separate distribution of important 
colors 13. When the set of palette colors 16 is determined based on this 
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supplemented distribution of colors 14, more palette colors will be assigned in the 
skin-tone region of color space, and therefore the quantization artifacts for these 
important colors will be reduced. 

The advantage of appending additional pixels to the image for the 
purpose of computing the supplemented distribution of colors 14 rather than 
simply combining the histogram of important colors and the histogram of input 
colors is that this method can be implemented without needing to modify the 
algorithm that is used to determine the set of palette colors. For example, if the 
algorithm were only available as a compiled executable software module, then it 
would be possible to practice the method of the present invention by simply 
including a pre-processing step of appending the additional image pixels to the 
input image before running the software module, and a post-processing step to 
strip the additional pixels off of the processed output image created by the 
software module. 

There are many different methods that could be used for the 
determine set of palette colors step 15. In a preferred embodiment of the present 
invention, a sequential scalar quantization technique such as that described by 
Allebach et al. in U.S. Patent 5,544,284 is used. This approach works by 
sequentially partitioning the colors of the supplemented distribution of colors 14 
into a set of color space regions. This is generally done on a luminance- 
chrominance representation of the image. The set of palette colors 16 is then 
determined by selecting an output color for each color space region in the set of 
color space regions. For more details of this method, reference should be made to 
the above-mentioned U.S. Patent 5,544,284. 

Other methods that could be used for the determine set of palette 
colors step 15 include a wide variety of vector quantization techniques. Several 
examples of these methods are discussed in U.S. Patent 5,544,284. It will be 
obvious to one skilled in the art that the method of the present invention can easily 
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be used with any image palettization technique where the selected palette colors 
are dependent on the distribution of colors in the input image. 

Once the set of palette colors 16 is determined, the assign palette 
color to each image pixel step 17 is used to form an output digital color image 18. 
One manner in which this step can be implemented is to determine the palette 
color having the smallest color difference relative to the color of each pixel of the 
input digital color image. 

In some cases, a particular input digital color image 10 might not 
contain any of the colors in the distribution of important colors 13. As a result, 
the method of the present invention would result in assigning palette colors that 
would never get used in output digital color image. 18. In a variation of the 
present invention, a test can be first applied to the input digital color image 10 to 
determine if it contains any colors in the distribution of important colors 13. If 
there are no such colors, then the step of forming the supplemented distribution of 
colors 14 can be skipped, and the set of palette colors can be determined directly 
from the distribution of input colors 12. One convenient means for determining 
whether the input digital color image 10 contains any colors in the distribution of 
important colors 13 is to examine the distribution of input colors 12. For example, 
if a three-dimensional histogram were used to represent the distribution of input 
colors 12, then it would simply be necessary to examine the histogram cell(s) 
corresponding to the colors in the distribution of important colors. If these 
histogram cell(s) were all found to contain zeros, then it can be concluded that the 
input digital color image 1 0 does not contain any colors in the distribution of 
important colors 13. If there are multiple subsets of important colors in the 
distribution of important colors 13 (for example skin-tones and neutrals), then 
only the subset(s) of important colors that are present in the distribution of input 
colors 1 1 need to be used to form the supplemented distribution of colors 14. In 
this case, the distribution of important colors is only used to supplement the 
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distribution of colors in the input digital color image in important color regions 
where the input digital color image contains a significant number of pixels. 

In a variation of the present invention, the assign palette color to 
each image pixel step 17 includes the application of a multi-level halftoning 
algorithm. Multi-level halftoning algorithms can be used to create the appearance 
of color values intermediate to the palette colors by varying the palette values 
assigned to the pixels of the output digital color image 1 8 such that the local mean 
color value is approximately preserved. An example of a multi-level halftoning 
method that could be used would be multi-level vector error diffusion. A flow 
diagram illustrating a typical multi-level vector error diffusion algorithm is shown 
in FIG. 3. In this figure, an input pixel color value I io from the i* column and j * 
row of the input digital color image 10 is processed by an assign palette color to 
each image pixel step 17 to form a corresponding output pixel color value of 
the output digital color image 18. The assign palette color to each image pixel 
step 1 7 introduces a quantization error due to the fact that the output pixel value is 
selected to be one of the palette colors in the determined set of palette colors 16. 
A difference operation 30 is used to compute a color error E io representing the 
vector difference between the input pixel color value 1^ and the output pixel color 
value Ojj. A weight errors step 32 is used to apply a series of error weights Wy to 
the resulting color error Ey. A sum operation 34 is then used to add the weighted 
color errors to nearby input pixels that have yet to be processed. An example set 
of error weights Wg is shown in FIG. 4. In this example, the color error for the 
current pixel 40 with column and row address (i, j) is weighted by a factor of V* 
and distributed to the next pixel to the right 42 in the current row of the image 
having the column and row address (i+1, j). Likewise the color error Ey is also 
weighted by factors of l A and distributed to three pixels in the next row of the 
image 44 having column and row addresses (i-1, j+1), (i, j+1) and (i+1 J+1). In 
this way, the quantization errors introduced when processing the current pixel 40 
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are distributed to nearby input pixels that have not yet been processed. The result 
is that the local mean color value is approximately preserved. 

A computer program product may include one or more storage 
medium, for example; magnetic storage media such as magnetic disk (such as a 
floppy disk) or magnetic tape; optical storage media such as optical disk, optical 
tape, or machine readable bar code; solid-state electronic storage devices such as 
random access memory (RAM), or read-only memory (ROM); or any other 
physical device or media employed to store a computer program having 
instructions for controlling one or more computers to practice the method 
according to the present invention. 

The invention has been described in detail with particular reference 
to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 
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